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Description 

The present invention relates to a method for operating a display device. In par- 
ticular, the present invention relates to a method for operating a display device 
wherein the graphical output is varied with respect to its content in dependence 
of the position or distance of a possible user. 

5 

Nowadays, a large variety of electronic equipment involves graphical user inter- 
faces to enable a possible user a particular convenient and reliable interaction 
with the electronic equipment. Therefore, a display device is provided on which 
e.g. information regarding the different operation modes and statuses can be 
10 displayed to inform a possible user with respect to the functional capabilities of 
the respective electronic equipment. 

( 

Known display devices display their display information in a constant manner, 
whereas the user of such an equipment often tends to move around and to 
15 change his position and /or orientation with respect to the display device and in 
particular with respect to the respective display unit of the display device. 

Under these varying positions and orientations it is not always possible and 
convenient for the user to percept the displayed information and its details. 
20 Therefore, it is necessary for the user to be positioned in a certain range and 
certain distance with respect to the electronic equipment, thereby reducing the 
flexibility and convenience for the user. 

It is an object of the present invention to provide a method for operating a display 
25 device which enables a particular convenient and reliable perception of displayed 
information by a user in particular independently from the user's position and/or 
orientation with respect to the display unit of the display device. 

The object is achieved by a method for operating a display device according to the 
30 characterizing features of independent claim 1. Additionally, the object is 
achieved by a method for operating the man-machine interface unit, by an appa- 
ratus, by a computer program product and by a computer readable storage me- 
dium acjcprding to the characterizing features of indepeiident claims 9. 10, 11, 
and 12. respectively. Preferred embodiments of the method for operating a display 
35 device are within the scope of the dependent sub-clsoms. 
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The method for operating a display device according to the present invention, in 
particular within a graphical user interface unit, comprises the steps of generat- 
ing and /or receiving user position information of a possible user in relation to an 
involved display unit of said display device, selecting and /or changing a display 
5 mode for displaying display information on said display unit and /or said display 
information itself, taking into account said user position information, displaying 
said display information or a derivative thereof on said display unit using said 
selected and /or changed display mode for said display unit and /or said selected 
and /or changed display information or a derivative thereof, thereby enabling a 
10 particular convenient and /or reliable perception of said displayed display infor- 
mation by said possible user in relation to the position of the user with respect to 
said display unit. 

It is, therefore, a basic aspect of the present invention to change and/or select a 
15 display mode or the information to be displayed, namely the display information, 
in dependence of generated and/or received user position information. Therefore, 
the position of the user with respect to the display unit is checked. It is decided 
in which display mode the display information and/ or what display information 
has to be or can be displayed on said display unit. Therefore, the process of 
20 displaying the display information can be adapted in accordance with the respec- 
tive position of the user compared to the position of the display unit and in ac- 
cordance to its variation. Therefore, in contrast to known display devices and 
methods for operating the same, the perception of displayed information by a 
possible user can be enhanced and improved. 

; 25 

In the sense of the invention the term "position" shall mean spatial localization, 
spatial orientation and/ or spatial distance. 

Therefore, according to a preferred embodiment of the method for operating a 
30 display device said user position information is designed to describe one or an 
arbitrary plurality or combination of the following aspects: a distance of the user 
and in particular the user's eyes with respect to said display unit, a first orienta- 
tion as an orientation of the users of the user's eyes location with respect to said 
display unit, in particular describing a view angle of the user vsrith respect to said 
35 display unit, a second orientation or a torsional orientation between the view axis 
of the user or the user's eyes and the display axis of said display unit. 

Further preferably, the step of generating and/ or receiving said user position 
information involves a process of measuring the distance between the possible 
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user or the possible user's eyes and the display unit. 

Additionally or alternatively, a distance or position sensing means is used for 
measuring the distance between a possible user or the possible user's eyes and 
5 the display unit. 

In this case, an ultrasonic sensor means, an infrared sensor means, a camera 
device and/ or the like may be involved. Additionally, any combination or plurality 
of these measures can be taken to realize said position sensing means. In the 
10 case of using a camera device an image processing means and /or an image /face 
recognition process can be adopted so as to obtain the position information from 
\ the user's face, for instance, the eyes of the user, e.g. from the orientation of a 

connection line between the user's eyes. 

15 According to a further alternative of the present invention by selecting and /or 
changing said display mode and /or said display information itself one or any 
arbitrary combination or plurality of the following aspects may be realized: 

- the size of the image and /or of parts thereof are adapted, 

20 - the resolution of the image and /or of parts thereof are adapted, 

- the representation of details of the image and/ or of parts thereof is adapted, in 
pcirticular with respect to the amount, the size, the color, 

- the view £m.gle of the user is compensated, 

- the torsional orientation between the view axis of the user and the display axis 
25 of the display unit is compensated, 

' the sem£mtic contents of the image and /or of parts thereof are adapted. 

Further additionally or alternatively, with increasing distance between a possible 
user and the display unit one or any combination or plurality of the following 
30 aspects may be realized: 

- with respect to text information the font size and /or the line width are in- 
creased, in particular in a continuous manner, 

- with respect to text information the amount of text is reduced and/ or only 
35 respective comparable most important information contents are chosen for the 

step of displa37ing, in particular by performing a process of re-phrasing, 

- with respect to image information the amount of details to be displayed is 
reduced, in particular in a continuous manner. 
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According to a further preferred embodiment it is provided that with decreasing 
distance between a possible user and the display unit one or any arbitrary com- 
bination or plurality of the following aspects may be realized: 

- with respect to text information the font size and /or the line width are de- 
5 creased, in particular in a continuous manner, 

- with respect to text information the amount of text is increased and /or respec- 
tive comparable less important information contents are chosen for the step of 
displaying, in particular by performing a process of re -phrasing, 

- with respect to image information the amount of details to be displayed is 
10 increased, in particular in a continuous manner. 

^ According to a further aspect of the present invention a method for operating a 

man-machine interface unit and in particular for operating a graphical user 
interface unit is provided, which comprises the inventive method for operating a 
15 display device. 

An additional aspect of the present invention is to provide an apparatus, in par- 
ticular a graphical user interface unit or a man-machine interface unit, which is 
adapted to realize the inventive method for operating a display device and /or to 
20 realize a method for operating a man-machine interface unit. 

According to a further aspect of the present invention a computer program prod- 
uct is provided comprising computer program means being adapted to realize the 
inventive method for operating a display device and/ or for realizing the inventive 
f 25 method for operating a man-machine interface unit when it is executed on a 
computer, a digital signal processing means and/or the like. 

Furtheron, according to the present invention a computer readable storage me- 
dium is provided which comprises the inventive computer program product. 

30 

These and further aspects of the present invention are further elucidated based on 
the following remarks: 

This invention inter alia relates to a graphical user interface and to a method for 
35 operating the same where the size, color, and/or semantic content of the dis- 
played information can be adjusted based on the position, distajace and /or orien- 
tation of the user with respect to a display unit and in particular to a screen. 
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Motivation 

In contrast to typical office applications, where the user sits in front of his PC, 
interaction with non-office information processing devices using embedded dis- 
5 plays is not necessarily performed from a fixed distajice. Consider everyday de- 
vices such as air conditioners, telephone answering machines and so on, where 
the user tends to glance at the display to get a quick feedback on the state of the 
machine from different positions relative to the device. It would be highly desir- 
able for the user to be able to read the most important information provided by 
10 such a device while passing it at a distance, yet be able to see more details while 
interacting with it at a close distance. 

Concepts 

15 The basic idea behind the notion of distance aware user interfaces is the reliance 
on a sensor device to measure the distance of the user from the screen. Instead of 
consciously changing the visualization through traditional user interaction wid- 
gets such as sliders, the user changes the visualization of the content based on 
his body movement. Similar to the way humans perceive features in nature that 

20 are located at different distances from the person through famlliax effects such as 
perspective changes in size, subdued colors observed on distance objects, and the 
reduction of information to a more outline based form for distance objects, the 
proposed user interface relies on similar techniques to adapt its content to the 
users viewing position. 

25 

SIZE : The size of visual artifacts on screen, such as the font size of text or the 
line width for graphics, are increased as the viewing distance of the person in- 
creases. The transformation is performed continuously in order to achieve a 
smooth transition from one size to the next. Features of the visualization that 
30 cannot be recognized anymore by the user due to the large viewing distance are 
dropped or changed into another visual representation that can still fit the size of 
the screen. 

COLOR : Features of the visualization that are put into the foreground due to 
35 viewing distance and/ or selection by the user are rendered with more saturated 
colors. Other features in the background are shown with less saturated colors. 
This effect re-enforces the Illusion of distance and helps the user to focus on the 
important aspect of the visualization. 
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INFORMATION REDUCTION : Textual output can be reduced in size by re- 
phrasing a given content in more or less verbose forms. This involves the creation 
of text from a semantic representation kept by the system that reflects the mean- 
ing of the content to be conveyed to the user. The result of this reduction of 
5 verbosity can be compared to the techniques used by the print media, which uses 
titles, short abstracts, and full text to present a given content at different levels of 
detail. 

Implementation 

10 

Two implementations of distance sensing have been successfully tried. The first 
uses video picture analysis, the second uses a special infrared sensor. Both 
methods are known in the literature or are incorporated into existing products. 
Besides the two mentioned methods that are further detailed below, further 
15 known methods of measuring distance are possible, e.g. measuring the time that 
an ultrasonic sound wave takes when being reflected at the user, e.g. used in 
parking aids, electronic yardstick), RF tag distance, laser interferometry. 

VIDEO ANALYSIS : The distance between the left and right eye is roughly constant 
20 at about 60 mm for all humans, children having a somewhat smaller distance. By 
detecting the face in a video picture shot by a camera mounted on top of the 
display, it can be inferred how far away the eye pair is from the camera by the 
distance of the detected eye balls in the picture through elementary geometric 
calculations. Video tracking of faces and eyes is obviously known, it is unclear 
25 whether the use of eye ball distance for distance estimation is known or not. 

INFRARED SENSOR : IR or infrared sensors can be used as long distance measuring 
sensors. 

30 Potential Applications 

Applications of the above described distance aware graphical user interface range 
from devices at home, personal portable devices, to public kiosk like setups. In 
particular the following applications scenarios can be considered: 

35 

1. Information display on a TV set such as for electronic program guides or EPG. 

2. IntelUgent home appliances with embedded displays such as air conditioners, 
weather stations, home security systems etc. 

3. Telephone answering machines.. 
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4. Portable devices such as PDAs and mobile phone that are visually monitored 
from a distance, e.g. when put on the desk top. 

6. Information kiosks at public places such as at train stations, airports, and 
museums. 

5 

In the following, the invention will be described in more detail taking reference to 
the accompanying figures on the basis of preferred embodiments of the invention. 

Fig. 1 is a flowchart describing a first embodiment of the inventive method 

10 for operating a display device. 

Fig. 2A, B demonstrate the dependence of the displayed information on the 
viewing distance of a user which can be involved in an embodiment of 
the inventive method for operating a display device. 

15 

Fig, 3 is a schematical block diagram describing a further embodiment of the 

inventive method for operating a display device. 

Fig, 4A, B elucidates a process for detecting a distance of an object with respect 
20 to a display unit by a triangulation method. 

> 

In the following, same elements and functions are indicated by the same reference 
symbols, and their detailed description is not repeated for each occurrence 
thereof. 

25 

Fig. 1 is a flowchart of a first embodiment of the inventive method for operating a 
display device DD. In a preliminary step SO of the embodiment shown in Fig. 1 the 
process is initialized and started. Then, in a first step SI the user position infor- 
mation UPI is generated and /or received. The receipt of the user position infor- 
30 mation UPI may be realized by connecting to an external measuring device. Alter- 
natively, the user position information UPI may also be generated by the method 
itself by performing a measuring process within step SI . 

In the following step 82 the display mode for the display unit DU is selected 
35 and/or changed. Also, in step S2 the display information DI may be selected 
and /or changed so as to derive a derivative DF of the display information DI. In 
the following step S3 said display information DI, DI' is displayed on a display 
unit DU. Then, in the following step S4 it is checked, whether the processing 
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session is ended or not. If the end is not reached then the process continues and 
refers back to the first step SI of the method shown in Fig. 1. 

In Figs. 2A and 2B the dependence of the displayed display information DI upon 
5 the viewing distance or the distance dU of the possible user U is explained. In Fig. 
2B the possible user is situated in a comparable small distance dU with respect to 
the display device DD of the display unit DU of the display device DD. Therefore, 
the full informational content of the display information DI is displayed, showing 
the temperature, the humidity and the action of a fan device. 

10 

In contrast, in Fig. 2A the possible user U is situated in a comparable large dis- 
tance dU with the respect to the display device DD or the display unit DU of the 
display device DD. Therefore, in the situation of Fig. 2A only a derivative DF of the 
display information DI of Fig. 2B is displayed on the display unit DU. now con- 
15 taining only a reduced informational content, namely the temperature ajnd the 
humidity only, and in a representation increased in size to be better readable in 
the distance dU of the use U. 

Fig. 3 demonstrates a more concrete embodiment of the present invention. Here, a 
20 possible user U is situated at a distance dU with respect to a display device DD 
comprising a display unit DU in the form of a personal computer. To obtain the 
distance dU between the possible user U and the display unit DU said display 
device DD comprises a position sensing means PS which can be referred to as a 
distance sensor PS. Based on a distance measurement with respect to the dis- 
25 tance dU performed in a step Tl in Fig. 3 by said distance sensor PS in a following 
step T2 said distance value dU is further processed and set into a visualization 
algorithm, upon the action of which the content and the representation of the 
display information DI is rendered within a further step T3 in Fig. 3. Finally, the 
process of displaying said display information DI is adapted. 

30 

In Figs. 4A and 4B a possible measuring process as indicated by the steps SI of 
Fig. 1 and Tl of Fig. 3 is explained in more detail. In Fig. 4A an object, namely a 
possible user U is situated comparable far at a comparable large distance dU 
before the display device DD having the display unit DU. In contrast, in the 
35 situation shown in Fig. 4B the object, namely the user U is situated comparable 
close at a comparable low distance dU before said display device DD and said 
display unit DU. In each case, a beam of light is emitted from the display unit DU 
and the angle of incidence with respect to a given point of reflection onto the 



- 10 - 



Sony International (Europe) GmbH 



object or user U is measured based on which angle the distances dU can be 
calculated. 
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List of Reference Symbols 

DD display device 

DI display information 

DI' derivative of display information, changed display information, selected 
display information 
5 DU display unit 

dU distance of an object /possible user 
GUI graphical user interface 
PS position sensing means, distance sensor 
PU location of a possible user /of the eyes of a possible user 
10 U user/ possible user 



